For nonvolatile semiconductor memory devices, microfabrication of memory cells is being pushed forward to increase the storage capacity. However, the memory cell characteristic problematically degrades along with the microfabrication of the memory cells. In, for example, a flash memory, the leakage current from an insulating layer in contact with a charge storage layer needs to be sufficiently small. This poses a limitation to thinning of a first insulating layer between a semiconductor layer and the charge storage layer and a second insulating layer between the charge storage layer and a control gate electrode. In addition, when the memory cells become finer, the capacitance (inter-cell interference) between adjacent memory cells cannot be neglected. For the purpose of reducing the inter-cell interference and increasing the coupling ratio, a structure in which a control gate electrode covers the side surface of a charge storage layer or a structure in which a second insulating layer uses a High-k material having a dielectric constant higher than that of SiO2 is used. However, both structures do not suffice for suppressing the degradation of the memory cell characteristic caused by microfabrication of memory cells.